Using the Optimized Combination Technique for Regression Problems

We present a generalisation of the sparse grid combination technique for regression in moderately high dimensions d ≤ 15. In contrast to the original combination technique the coefficients in the combination formula do not depend only on the used partial grids, but instead on the function to be reconstructed, i.e., on the given data. The coefficients are computed to fulfill a certain optimality condition in a projection sense, first proposed in [2]. With this modified ansatz we are able to address instability issues of the normal combination technique, which were observed for some real life data sets in [1]. We will present results for a range of benchmark data sets for regression showing the feasibility of this new ansatz in comparison to the normal combination technique as well other standard algorithms for regression. Furthermore we will report on theoretical studies of the instability of the combination technique and will relate that to angles between the employed discrete spaces. Besides giving results which compare the error of the normal combination technique with the optimized combination technique using the (potentially large) angles between spaces, we will show in experiments under what conditions these angles can be large for regression problems.